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ABSTRACT 


Research  on  '“Understanding  Objects,  Features,  and 
Backgrounds,'5;  carried  out  during  the  period  May  1980  - 
December  1982,  is  summarized  under  the  following  headings 

a)  Object  segmentation  and  recognition 

b)  Feature  extraction  and  background  analysis 

c)  Multi-resolution  image  analysis 

d)  Time-varying  imagery  analysis 

Further  details  can  be  found  in  the  32  technical  reports 
that  were  issued  on  the  project;  a  list  of  these  reports, 
together  with  their  abstracts,  is  provided. 
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Status  reports 


1 .  Introduction 

in  June  1976  the  U.S.  Army  Night  Vision  and  Electro-Optics 
Laboratory  awarded  Contract  DAAG-5 3-71-C-0138  to  the  University 
of  Maryland  for  research  on  "Algorithms  and  Hardware  Technology 
for  Image  Recognition."  Funding  for  this  contract  was  derived 
primarily  from  the  Defense  Advanced  Research  Projects  Agency 
under  DARPA  Order  3206.  During  the  following  21-month  period, 
the  University  developed  and  tested  advanced  algorithms  for 
detection  of  tactical  targets  on  Forward-Looking  InfraRed  (FLIR) 
imagery.  Concurrently,  on  a  subcontract,  the  Westinghouse 
Defense  Systems  Division  designed  charge-coupled  device  (CCD) 
layouts  for  implementing  many  of  these  algorithms  in  hardware, 
and  also  fabricated  a  CCD  chip  that  implemented  one  basic  algo¬ 
rithm,  histogramming/sorting.  The  results  of  the  work  done 
during  the  first  21  months  of  the  contract  are  documented  in 
detail  in  a  Final  Report  dated  March  1978  [Al].* 

In  April  1978  the  contract  was  extended  for  a  two-year 
period,  under  the  new  title  "Image  Understanding  Using  Overlays." 
During  this  phase  of  the  project,  numerous  algorithms  were  de¬ 
veloped  and  tested  for  object  detection  and  extraction  from 
images,  as  well  as  for  image  and  region  representation.  On  a 
subcontract,  Westinghouse  investigated  the  implementation  of 
some  of  these  algorithms  in  general-  or  special-purpose  digital 
hardware.  Westinghouse  also  conducted  tests  of  one  class  of 
algorithms  known  as  "relaxation"  techniques.  The  results  of 

♦Numbers  in  brackets  refer  to  the  report  abstracts  given  in  the 
Appendix. 


the  work  done  during  this  period  are  documented  in  a  series  of 
technical  and  semiannual  reports,  and  are  summarized  in  a 
Final  Report  dated  May  1980  [A2]. 

In  May  1980  the  contract  was  extended  for  a  final  two-year 
period  (later  extended,  at  no  additional  cost,  through  December 
1982),  under  the  title  "Understanding  Objects,  Features,  and 
Backgrounds."  During  this  phase  of  the  project,  further 
studies  were  conducted,  in  collaboration  with  Westinghouse ,  on 
object  segmentation  and  recognition,  feature  extraction  and 
background  analysis,  multi-resolution  image  processing  tech¬ 
niques,  and  analysis  of  time-varying  imagery.  This  work  was 
documented  in  a  series  of  project  status  reports  [Bl-3]  and 
Technical  Reports  [Cl-32] ,  and  is  summarized  in  this  Final 
Report. 

Principal  Investigators  on  this  project  at  the  University 
of  Maryland  were  Profs.  Azriel  Rosenfeld  and  Larry  S.  Davis, 
and  at  Westinghouse,  Dr.  Glenn  E.  Tisdale  and  Mr.  Bruce  J. 
Schachter.  The  project  monitor  at  NVEOL  is  Dr.  George  R.  Jones. 


Object  segmentation  and  recognition 


a)  Comparative  segmentation  study 

£  comparative  study  of  object  extraction  techniques 
applicable  to  FLIR  imagery  was  conducted  jointly  by 
Maryland  and  Westinghouse,  using  a  database  of  52 
images  collected  by  Westinghouse  from  Army,  Navy,  and 
Air  Force  sources.  Techniques  tested  by  Maryland  in¬ 
cluded  two  variations  of  a  relaxation  method  as  well 
as  new  methods  based  on  multiresolution  image  represen¬ 
tations,  known  as  "pyramids."  One  of  the  pyramid-based 
methods  outperformed  all  the  other  techniques  tested. 

The  results  of  the  Maryland  study  are  documented  in  de¬ 
tail  in  a  technical  report  [C19] ,  while  Westinghouse' s 
study  is  documented  in  a  Westinghouse  report. 

b)  New  segmentation  techniques 

As  a  supplement  to  the  main  segmentation  study, 
several  new  segmentation  techniques  were  developed  under 
the  project.  Two  methods  developed  on  earlier  projects 
were  extended  from  single-band  to  multiband  imagery. 

One  of  these  improves  the  detectability  of  clusters  in 
a  histogram  or  scatterplot  by  suppressing  pixels  that 
lie  on  edges  [Cl].  The  other,  known  as  "Superspike," 
converts  the  peaks  in  a  histogram  or  scatterplot  into 
sharp  spikes  (thus  making  them  trivially  detectable)  by 
a  process  of  iterated  local  averaging  in  which  the  histo¬ 
gram  is  used  as  a  guide  in  selecting  those  neighbors  with 


which  a  given  pixel  should  be  averaged  [C26] .  A  third 


method,  "bimean  clustering,"  identifies  the  two  "best" 
subpopulations  in  a  histogram  by  finding  the  pair  of 
values  that  gives  a  best  fit  to  the  histogram  in  the 
least  squares  sense  [C20] . 

c)  Object  identification  using  constraint  filtering 
The  conventional  approach  to  recognizing  targets 
in  FLIR  imagery  is  to  extract  potential  target  regions 
using  segmentation  techniques,  and  then  carefully  ana¬ 
lyze  the  properties  of  each  region  independently  in 
order  to  determine  whether  or  not  it  could  be  a  target. 

We  have  investigated  a  complementary  approach  based 
on  comparisons  among  regions  rather  than  analysis  of  in¬ 
dividual  regions.  After  the  image  is  segmented,  we 
give  each  region  a  set  of  possible  labels  -  e.g.,  "sky," 
"ground,"  "smoke,"  "tree,"  "tank."  We  then  attempt 
to  eliminate  labels  from  the  regions  based  on  their  rela¬ 
tionships  with  other  regions  (relative  property  values, 
relative  posit  ions ,  etc. )  .  This  method  performed  success¬ 
fully  in  a  small  set  of  tests;  it  eliminated  the  "tank" 
label  from  all  the  non-tank  regions  but  kept  it  for  all 
the  tank  regions  [C25] .  This  approach  should  be  of 
interest  as  a  supplement  to  existing  target  recognition 
algorithms . 


Feature  extraction  and  background  analysis 


a )  Edge  and  corner  extraction 

Feature  detection  (e.g.,  edge  detection)  is  an  impor 
tant  adjunct  to  object  recognition,  and  also  plays  an 
important  role  in  image  matching  (e.g.,  for  object  track 
ing  and  time-varying  imagery  analysis) .  Three  feature 
detection  studies  were  conducted  on  this  project.  The 
optimal  approach  to  edge  detection  developed  by  Kueckel, 
which  finds  the  best-fitting  step  function  to  a  given 
image  neighborhood,  was  applied  to  derive  optimal  edge 
operators  for  a  class  of  small  neighborhoods  [C28] .  A 
basic  new  method  of  evaluating  edge  detector  output, 
based  on  consistency  of  the  edge  output  data,  was 
developed  and  successfully  tested  [C8] .  A  simplified 
method  of  corner  detection  was  developed  based  on  detect 
ing  discontinuities  in  one-dimensional  projections  of 
the  image;  this  method  eliminates  the  need  to  apply  com¬ 
putationally  expensive  higher-order  derivative  operators 
at  every  point  of  the  image  [ C 1 3 ] . 

b)  Blob  and  ribbon  extraction 

Work  was  also  done  on  the  detection  of  higher-level 
features  such  as  "blobs"  and  "ribbons"  in  an  image.  (A 
blob  is  surrounded  by  consistently  facing  edges,  while 
a  ribbon  is  characterized  by  "antiparallel,"  oppositely 
facing  edges.)  Edge  linking  schemes  were  developed  for 
detecting  such  features  based  on  compatibi) iuy  of  the 


edges  with  respect  to  both  geometry  and  gray  level  [C2] 


Quantitative  measures  for  edge  compatibility  were  also 
developed  for  assessing  both  closedness  [C3]  and  anti¬ 
parallelness  C  C4 )  . 
c)  Texture  analysis 

In  connection  with  image  background  characterization, 
two  texture  analysis  studies  were  conducted.  An  approach 
to  texture  analysis  based  on  average  strength  of  match 
with  various  local  patterns  was  implemented;  it  was  found 
to  perform  better  than  several  standard  methods  [C18]. 

The  idea  of  applying  texture  measures  to  arrays  of  terrain 
elevation  data  was  also  briefly  explored;  if  such  data 
were  available  at  sufficient  resolution,  it  would  provide  a 
useful  supplement  to  intensity-based  texture  analysis  [C15] . 


Feature  extraction  and  background  analysis 

a)  Edge  and  corner  extraction 

Feature  detection  (e.g.,  edge  detection)  is  an  impor 
tant  adjunct  to  object  recognition,  and  also  plays  an 
important  role  in  image  matching  (e.g.,  for  object  track 
ing  and  time-varying  imagery  analysis) .  Three  feature 
detection  studies  were  conducted  on  this  project.  The 
optimal  approach  to  edge  detection  developed  by  Hueckel, 
which  finds  the  best-fitting  step  function  to  a  given 
image  neighborhood,  was  applied  to  derive  optimal  edge 
operators  for  a  class  of  small  neighborhoods  [C28] .  A 
basic  new  method  of  evaluating  edge  detector  output, 
based  on  consistency  of  the  edge  output  data,  was 
developed  and  successfully  tested  [C8] .  A  simplified 
method  of  corner  detection  was  developed  based  on  detect 
ing  discontinuities  in  one-dimensional  projections  of 
the  image;  this  method  eliminates  the  need  to  apply  com¬ 
putationally  expensive  higher-order  derivative  operators 
at  every  point  of  the  image  [C13]  . 

b)  Blob  and  ribbon  extraction 

Work  was  also  done  on  the  detection  of  higher-level 
features  such  as  "blobs"  and  "ribbons"  in  an  image.  (A 
blob  is  surrounded  by  consistently  facing  edges,  while 
a  ribbon  is  characterized  by  "antiparallel,"  oppositely 
facing  edges.)  Edge  linking  schemes  were  developed  for 
detecting  such  features  based  on  compatibility  of  the 
edges  with  respect  to  both  geometry  and  gray  level  [ C 2 ] . 


Quantitative  measures  for  edge  compatibility  were  also 
developed  for  assessing  both  closedness  [ C 3 )  and  anti¬ 
parallelness  [C4]. 
c )  Texture  analysis 

In  connection  with  image  background  characterization, 
two  texture  analysis  studies  were  conducted.  An  approach 
to  texture  anal^  sis  based  on  average  strength  of  match 
with  various  local  patterns  was  implemented;  it  was  found 
to  perform  better  than  several  standard  methods  [C18]. 

The  idea  of  applying  texture  measures  to  arrays  of  terrain 
elevation  data  was  also  briefly  explored;  if  such  data 
were  available  at  sufficient  resolution,  it  would  provide  a 
useful  supplement  to  intensity-based  texture  analysis  [ Cl 5 ] . 


Multi-resolution  image  analysis 

a )  Background  and  related  work 

A  potentially  powerful  new  approach  to  image  analysis, 
now  under  development  at  our  laboratory,  is  based  on  the 
use  of  a  "pyramid"  of  successively  reduced-resolution  ver¬ 
sions  of  the  given  image.  Initial  work  on  image  segmenta¬ 
tion  using  pyramids  was  done  under  NSF  sponsorship.  Dur¬ 
ing  the  summer  of  1982,  a  workshop  on  "Multiresolution 
Image  Processing  and  Analysis"  was  held,  also  under  NSF 
sponsorship,  at  which  about  25  research  groups  presented 
recent  results  that  make  use  of  multiresolution  image  rep¬ 
resentations  in  various  ways.  The  pyramid  image  represen¬ 
tation  also  has  the  advantage  of  compatibility  with  the 
quadtree  region  representation,  which  was  extensively  stu¬ 
died  during  an  earlier  phase  of  this  project,  and  which  is 
being  further  studied  in  connection  with  cartographic  data 
base  applications  under  the  sponsorship  of  the  U.S.  Army 
Engineer  Topographic  Laboratory. 

b)  Segmentation  and  representation 

One  way  of  using  the  pyramid  representation  in  image 
segmentation  is  to  define  links  between  pixels  and  their 
"parents"  at  consecutive  levels  of  the  pyramid,  based  on 
mutual  similarity;  this  gives  rise  to  subtrees  of  the 
pyramid,  and  thus  defines  a  partition  of  the  image, 
where  each  part  consists  of  the  pixels  that  are  the  leaves 
of  a  given  subtree.  A  number  of  variations  on  this  basic 
approach  were  investigated  [C6],  and  it  was  also  generalized 


to  multispectral  imagery  [Cll] .  In  connection  with 
quadtree  region  representation,  earlier  work  on  the 
generation  of  an  image  row  by  row  from  its  quadtree 
was  extended  to  include  several  new  algorithms  [C7] 
c)  Feature  extraction  and  encoding 

Pyramids  can  also  be  used  to  extract  and  represent 
features  such  as  edges  and  blobs  in  an  image.  If  we  use 
mutual  similarity  as  a  basic  for  linking  "edgels,"  rather 
than  pixels,  in  a  pyramid  representation,  we  obtain 
"trees"  of  edges  which  allow  us  to  detect  the  major  edges 
in  an  image,  at  the  higher  levels  of  the  pyramid,  and 
then  locate  these  edges  precisely  at  the  full-resolution 
level  ( Cl 4 ] .  Pyramids  can  also  be  used  to  encode  edges 
(or  curves)  detected  in  an  image,  yielding  successively 
coarser  approximations  as  long  as  the  eages  crossing  a 
given  block  of  the  image  can  be  compactly  approximated 
[C5].  These  approximations  can  then  be  used  as  an  aid  in 
linking  together  edge  segments  that  lie  on  long  lines 
or  smooth  curves  (C30).  Two  approaches  to  blob  extraction 
using  pyramids  were  also  investigated.  One  of  these  uses  pixel  linking 
to  construct  subtrees  of  the  pyramid  such  that  leaves  of  each 
subtree  are  the  pixels  that  belong  to  a  compact,  homogeneous 
piece  of  the  image  [C24]\  Another  approach  is  based  on  the 
fact  that  any  blob  shrinks  to  a  (local)  "spot"  at  some  level 
of  the  pyramid;  it  detects  blobs  by  constructing  an  edge 


pyramid  and  detecting  pixels  that  are  locally  surrounded  by 
edges  [C21] .  This  method  outperformed  all  the  others  that 
were  tested  in  the  comparative  study  of  FLIR  image  segmenta¬ 
tion  techniques  (see  above.  Section  2a). 


Time-varying  imagery  analysis 


a)  Image  matching 

One  approach  to  detecting  and  analyzing  motion  in 
an  image  sequence  is  to  identify  sets  of  corresponding 
points  in  successive  frames  of  the  sequence.  This  is 
usually  done  by  searching  for  matches  to  pieces  of  one 
frame  in  the  other  frame.  In  order  to  obtain  sharp 
matches,  it  is  desirable  to  use  pieces  that  contain 
distinctive,  high-contrast  features  such  as  corners 
(they  are  preferable  to  edges  because  the  match  to  an 
edge  is  insensitive  to  displacement  in  the  direction 
along  the  edge) .  Some  successful  experiments  in  image 
matching  using  corner  features  are  described  in  [C12] . 

A  supplemental  experiment,  reported  in  [C17],  showed 
that  local  intensity-based  matching  in  the  neighborhood 
of  a  feature  point  can  be  used  to  unambiguously  locate 
match  peaks  in  those  cases  where  the  results  of  feature 
matching  are  ambiguous. 

b)  Motion  estimation  and  smoothing 

Another  approach  to  motion  detection,  appropriate 
in  cases  where  the  rate  of  motion  does  not  exceed  one 
pixel  per  frame,  involves  using  the  space  and  time  deri¬ 
vatives  of  the  image  intensity  at  each  pixel  to  estimate 
a  motion  vector  at  that  pixel.  This  method  yields  reli¬ 
able  estimates  of  motion  components  only  in  directions 
where  there  are  rapid  changes  in  gray  level.  Thus  in  a 
smooth  region  it  yields  no  useful  information;  at  an  edge 


it  yields  only  the  component  of  motion  in  the  direction 


across  the  edge;  but  at  corner  pixels  it  yields  two 
components,  thus  allowing  the  entire  motion  vector  to 
be  estimated  C Cl 6 ] .  Given  a  region  in  the  image  rep¬ 
resenting  a  rigid  object  moving  parallel  to  the  image 
plane,  we  can  estimate  motion  vectors  at  the  corners  of 
the  object  and  "propagate"  these  estimates  around  the 
edges  of  the  object  to  determine  its  motion  (translation 
and  rotation) .  This  approach  to  motion  estimation  was 
developed  in  a  series  of  reports  [C22,C23,C29] . 

The  motion  vector  fields  obtained  from  small  image 
neighborhoods  are  noisy.  If  they  are  smoothed  by  simple 
local  averaging,  incorrect  results  are  obtained  at  the 
boundaries  of  moving  objects.  A  better  approach  is  to 
use  nonlinear  smoothing  techniques  based  on  selective 
local  averaging;  this  does  not  blur  sharp  edges  [C31] . 

A  related  problem  is  that  of  smoothing  the  images  in  a 
sequence  by  averaging  successive  frames;  here  one  cannot 
simply  average  corresponding  pixels,  but  must  introduce 
displacements  in  order  to  allow  for  the  motion.  In  this 
connection,  one  need  not  know  the  entire  motion  vector, 
but  only  its  component  in  the  gradient  direction,  since 
errors  in  the  tangential  direction  will  not  cause  edges 
to  become  blurred  [C32]. 
c)  Optical  flow  analysis 

The  changes  in  an  image  sequence  due  to  the  motion  of 
the  observer  relative  to  the  scene,  rather  than  to  object 
motion,  are  known  as  "optical  flow." 


Given  an  array  of 


motion  vectors  representing  optical  flow,  methods  have 
been  developed  of  inferring  the  parameters  of  the  ob¬ 
server's  motion  (translational  and  rotational)  and  of 
deriving  the  relative  distances  between  the  observer 
and  the  points  in  the  scene.  Algorithms  for  deriving 
relative  scene  distance  and  local  surface  orientation 
from  optical  flow  are  presented  in  [C9],  while  a  method 
of  deriving  the  observer's  instantaneous  direction  of 
motion  from  optical  flow,  and  of  decomposing  his  motion 
into  translational  and  rotational  components,  is  de¬ 
veloped  in  [CIO]. 


t 


6 .  Status  reports 

As  mentioned  in  the  Introduction,  three  project  status 
reports  were  issued  [Bl-3]  that  summarize  the  work  done 
during  this  phase  of  the  project.  The  first  and  third  of 
these  reports  were  also  published  in  the  Proceedings  of 
the  two  DARPA  Image  Understanding  Workshops  that  were  held 
during  this  period  (April  1981  and  September  1982) . 

At  a  meeting  of  the  principal  investigators  on  the 
DARPA  Image  Understanding  Program,  held  in  January  1982, 
it  was  decided  to  prepare  a  Final  Report  on  the  overall 
program.  The  University  of  Maryland  was  asked  to  draft 
the  portion  of  this  report  dealing  with  two-dimensional 
image  analysis  techniques  ("low-level  vision") .  An  edited 
version  of  this  dr^ft  was  also  issued  as  a  technical  report 
(C2 7 ]  . 


APPENDIX :  REPORT  ABSTRACTS 


1.  Final  report:  Algorithms  and  hardware  technology 
for  image  recognition,  March  1978. 

ABSTRACT:  Algorithms  for  detecting  and  classifying 

tactical  targets  on  forward-looking  infrared  (FLIR) 
imagery  were  developed.  A  Westinghouse  subcontract 
investigated  the  implementabil ity  of  these  algorithms 
in  charge-coupled  device  (CCD)  technology,  and  also 
successfully  implemented  one  basic  function,  sorting. 

2.  Final  report:  Image  understanding  using  overlays, 
May  1980. 

ABSTRACT:  This  report  summarizes  accompli sh-men cs  on 

the  project  during  the  period  1  Octcber  1979  -  30 
March  1980  under  three  headings:  (1)  segmentation 
and  texture  analysis;  (2)  local  and  global  shape  ana¬ 
lysis;  (3)  hierarchical  representation.  It  also 
describes  the  efforts  to  be  conducted  during  the  next 
two-year  phase  of  the  project,  under  the  title  "Under¬ 
standing  Features,  Objects,  and  Backgrounds,"  and  pro¬ 
vides  a  listing  of  all  the  reports  issued  and  papers 
published  during  the  two-year  phase  just  completed. 


Interim  Reports  on  the  current  phase  of  the  project 

1.  Semi-annual  report  (1  April  1980  -  31  January  1981), 
February  1981. 

ABSTRACT:  Current  activities  on  the  project  are  re¬ 
viewed  under  the  following  headings:  (1)  segmentation, 

(2)  local  feature  detection,  (3)  feature  linking,  and 
(4)  hierarchical  representation. 

2.  Semi-annual  report  (1  February  -  31  July  1981), 
August  1981. 

ABSTRACT:  Current  activities  on  the  project  are  re¬ 
viewed  under  the  following  headings:  (1)  segmentation, 

(2)  local  feature  detection  and  matching,  (3)  hier- 


3.  Project  status  report  {1  August  1981  -  31  July  1982) , 

July  1982. 

ABSTRACT:  Current  activities  on  the  project  are  summarized  under 

the  following  headings:  (a)  preprocessing  and  segmentation, 

(b)  feature  detection  and  texture  analysis,  (c)  hierarchical 
representations,  (d)  matching  and  motion. 


C.  Technical  Reports  on  the  current  phase  of  the  project 

1.  Alan  Broder  and  Azriel  Rosenfeld,  "Gradient  Magnitude  as  an 

Aid  in  Color  Pixel  Classification".  TR-906,  DAAG-5 J-76C-0138 , 
June  1980. 

ABSTRACT:  When  pixels  in  a  black-and-white  image  are  classified  by 
thresholding  their  gray  levels,  gradient  magnitude  information  can 
he  used  in  various  ways  as  an  aid  in  threshold  selection.  This 
note  deals  with  the  use  of  color  gradient  magnitude  as  an  aid  in 
classifying  pixels  based  on  their  colors  or  spectral  signatures. 

2.  Mohamad  Tavakoli  and  Azriel  Rosenfeld,  "Toward  the  Recognition 

of  Buildings  and  Roads  on  Aerial  Photographs".  TR-913, 
DAAG-53-76C-0138,  July  1980. 

ABSTRACT:  This  paper  describes  steps  toward  the  recognition  of 
cultural  features  such  as  buildings  and  roads  on  aerial  photographs. 
The  approach  involves  several  successive  stages  of  grouping  of 
edge  segments.  Straight  line  segments  are  fitted  to  sets  of  edge 
pixels;  compatibilities  between  pairs  of  these  segments,  based  on 
gray  level  and  geometric  information,  are  computed;  and  the  seg¬ 
ments  are  then  grouped  into  building-like  and  road-like  groupings 
based  on  those  compatibilities.  Examples  of  the  results  obtained 
vising  this  approach  are  given,  and  some  variations  on  the  initial 
stages  of  the  process  are  also  investigated. 


3.  Mohamad  Tavakoli  and  Azriel  Rosenfeld,  "A  Method  for  Linking 

Pairs  of  Compatible  Linear  Features".  TR-930,  DAAG-5 3-76C- 
0133,  August  1980. 

ABSTRACT:  A  method  of  linking  compatible  straight  edg^s  is  presented 

vin.i  discussed.  The  linking  is  based  on  information  about  the  geo¬ 
metrical  configuration  of  the  edges,  the  similarity  oi  tne  gray 
levels  on  their  object  sides,  and  the  similarity  of  their  object 
sides  with  the  line  joining  their  endpoints.  Three  fi-.  ;ru*s  of  merit 
are  defined  for  evaluating  pairs  of  segments  for  possible  Linking. 
Examples  are  shown  of  applying  the  method  to  high  resolution  aerial 
photographs.  Results  indicate  that  cultural  features  such  as  roads 
.•no  buildings  can  be  extracted  and  that  a  significant  reduction  in  the 
complexity  of  the  image  description  can  be  obtained.  This  approach 
snould  be  especially  useful  for  defining  the  degrees  of  compatibility 
•f  pairs  of  edges  in  a  relaxation  scheme  for  classifying  linear 
feature  segments. 


Copy 


4  . 


Mohamad  Tavakoli  and  Azriel  Rosenfela,  "A  Method  for 
Finding  Pairs  of  Anti-Parallel  Linear  Features." 

TR-943,  DAAG-53- 7 6C-0138 ,  September  1980. 

ABSTRACT:  A  method  for  finding  anti-parallel  straight  edges 

is  presented  and  discussed.  This  method  is  based  on  informa¬ 
tion  about  the  object  sides  of  edges,  the  similarity  and 
homogeneity  of  gray  level  between  the  edges,  the  angle 
difference,  the  amount  of  overlap,  and  also  information 
about  the  estimated  object  gray  level.  Two  figures  of 
merit  are  defined  to  calculate  the  mutual  support  of  two 
antiparallel  linear  features.  Examples  are  shown  of  apply¬ 
ing  the  method  to  high  resolution  aerial  photographs.  Re¬ 
sults  indicate  that  cultural  features  such  as  roads  and 
buildings  can  be  extracted  and  that  a  significant  reduction 
in  the  complexity  of  the  image  description  can  be  obtained. 

This  algorithm  should  be  especially  useful  in  a  relaxation 
type  scheme  for  classifying  linear  feature  segments  when 
the  degree  of  anti-parallelism  of  the  segments  is  needed. 

5.  Michael  Shneier,  "Two  Hierarchical  Linear  Feature 

Representations:  Edge  Pyramids  and  Edge  Quadtrees." 

TR-961,  DAAG-S3-76C-0138,  October  1980. 

ABSTRACT:  Two  related  methods  for  the  hierarchical  representa¬ 

tion  of  curve  information  are  presented.  First,  edge  pyramids 
are  defined  and  discussed.  An  edge  pyramid  is  a  sequence  of 
successively  lower  resolution  images,  each  image  containing 
a  summary  of  the  edge  or  curve  information  in  its  predecessor. 
This  summary  includes  the  average  magnitude  and  direction  in 
a  neighborhood  of  the  preceding  image,  as  well  as  an  inter¬ 
cept  in  that  neighborhood  and  a  measure  of  the  error  in  the 
direction  estimate.  An  edge  quadtree  is  a  variable-resolution 
representati  n  of  the  linear  information  in  the  image.  It  is 
constructed  !,/  recursively  splitting  the  image  into  quadrants 
based  on  magnitude,  direction  and  intercept  information.  Ad¬ 
vantages  of  the  edge  quadtree  representation  are  its  ability 
to  represent  several  linear  features  in  a  single  tree,  its 
registration  with  the  original  image,  and  its  ability  to 
perform  many  common  operations  efficiently. 

6.  Teresa  silberberg,  Shmuel  Peleg,  and  Azriel  Rosenfeld, 

"Multi- Resolution  Pixel  Linking  for  Image  Smoothing  and 

Segmentation."  TR-977,  DAAG-53-76C-0138 ,  November  1980. 

ABSTRACT:  When  an  image  is  smoothed  using  small  blocks  or 

neighborhoods,  the  results  may  be  somewhat  unreliable  due  to  the 
effects  of  noise  on  small  samples.  When  larger  blocks  are  used, 
t.r.c  samples  become  more  reliable,  but  they  are  more  likely  to  be 
mixed,  since  a  large  block  will  often  not  be  contained  in  a  single 
region  of  the  image.  A  compromise  approach  is  to  use  several 
clock  sizes,  representing  versions  of  the  image  at  several  reso¬ 
lutions,  and  to  carry  out  the  smoothing  by  means  of  a  cooperative 


i't'.'Ce ss  based  or;  links  between  blocks  Of  adjacent  sizes.  These 
links  lie: ine  "block  trees"  which  segment  the  image  into  regions, 
nt-t  iioccssar ily  connected,  over  which  smoothing  takes  place. 

I  r,  *  i .  i  s  paper ;  a  number  of  variations  on  the  basic  dock  linking 
approach  are  investigated,  and  some  tentative  conclusions  are 
drawn  regardin'.)  preferred  methods  of  initializing  the  process 
an!  defining  the  links,  yielding  improvements  over  the  ori¬ 
ginally  proposed  method. 

7.  Hanan  Samet,  "Algorithms  for  the  Conversion  of  Quadtrees  to 

Rasters."  TR-979,  DAAG-53-76C-0138 ,  November  1980. 

ABSTRACT:  A  number  of  algorithms  are  presented  for  obtaining  a 

raster  representation  for  an  image  given  its  quadtree.  The 
algorithms  are  given  in  an  evolutionary  manner  starting  with 
tne  straightforward  top-down  approach  that  visits  each  run  in 
a  row  in  succession  starting  at  the  root  of  the  tree.  The 
remaining  algorithms  proceed  in  a  manner  akin  to  an  inorder  tree 
traversal.  All  of  the  algorithms  are  analyzed  and  an  indica¬ 
tion  is  given  as  to  when  each  is  preferable.  The  execution  time 
of  all  of  the  algorithms  is  shown  to  be  proportional  to  the  sum 
of  the  heights  of  the  blocks  comprising  the  image. 

8.  Los  Kitchen  and  Azriel  Rosenfeld,  "Edge  Evaluation  Using 

Local  Edge  Coherence."  TR-981,  DAAG-53-76C-0138 , 

December  1980. 

ABSTRACT:  A  method  of  evaluating  edge  detector  output  is  pro¬ 
posed,  based  on  the  local  good  form  of  the  detected  edges. 

It  combines  two  desirable  qualities  of  well-formed  edges  — 
good  continuation  and  thinness.  The  measure  has  the  expected 
behavior  for  known  input  edges  as  a  function  of  their  blur 
and  noise.  It  yields  results  generally  similar  to  thos^  ob¬ 
tained  with  measures  based  on  discrepancy  of  the  detected  edges 
from  their  known  ideal  positions,  but  it  has  the  advantage  of 
not  requiring  ideal  positions  to  be  known.  It  can  be  used  as 
an  aid  to  threshold  selection  in  edge  detection  (pick  the 
threshold  that  maximized  the  measure) ,  as  a  basis  for  comparing 
the  performances  of  different  detectors,  and  as  a  measure  of 
the  effectiveness  of  various  types  of  preprocessing  operations 
facilitating  edge  detection. 

9.  K.  Prazdny,  "Relative  Depth  and  Local  Surface  Orientation 

from  Image  Motions."  TR-996,  DAAG-53-76C-0138 , 

January  1981. 

ABSTRACT:  A  simple  mathematical  formalism  is  presented  suggest- 

i ng  a  mechanism  for  computing  relative  depth  of  any  two  tex¬ 
ture  elements  characterized  by  the  same  relative  motion  parameters. 
The  method  is  based  on  a  ratio  of  a  function  of  the  angular 
velocities  of  the  projecting  rays  corresponding  to  the  two  tex¬ 
ture  elements.  The  angular  velocity  of  a  ray  cannot,  however, 
be  computed  directly  from  the  instantaneous  character ization 
of  motion  of  a  "retinal"  point.  It  is  shown  how  it  can  be 
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obtained  from  the  (linear)  velocity  of  the  image  element  on  th 
projection  surface  and  the  first  time  derivative  of  its  direc¬ 
tion  vector.  A  similar  analysis  produces  a  set  of  equations 
which  directly  yield  local  surface  orientation  relative  to  a 
given  visual  direction.  The  variables  involved  are  scalar 
quantities  directly  measurable  on  the  projection  surface  but, 
unlike  the  case  of  relative  depth,  the  direction  of  (instan¬ 
taneous)  motion  has  to  be  computed  by  different  means  before 
the  method  can  be  applied.  The  relative  merits  of  the  two 
formalisms  are  briefly  discussed. 

10.  K.  Prazdny,  "Determining  the  Instantaneous  Direction  of 

Motion  from  Optical  Flow  Generated  by  a  Curvilinear- 
ly  Moving  Observer."  TR-1009,  DAAG-53-76C-0138 , 
February  1981. 

ABSTRACT:  A  method  is  described  capable  of  decomposing  the 
optical  flow  into  its  rotational  and  translational  components. 
The  translational  component  is  extracted  implicitly  by  loca¬ 
ting  the  focus  of  expansion  associated  with  the  translational 
component  of  the  relative  motion.  The  method  is  simple, 
relying  on  minimizing  an  (error)  function  of  3  parameters.  As 
such,  it  can  also  be  applied  without  modification,  in  the  case 
of  noisy  input  information.  Unlike  the  previous  attempts 
at  interpreting  optical  flow  to  obtain  information  about  the 
three-dimensional  disposition  of  texture  elements,  the  method 
uses  only  relationships  between  quantities  on  the  projection 
plane.  No  3D  geometry  is  involved.  Also  outlined  is  a  pos¬ 
sible  use  of  the  method  for  the  extraction  of  that  part  of 
the  optical  flow  containing  information  about  relative  depth 
directly  from  the  image  intensity  values,  without  extracting 
the  "retinal"  velocity  vectors. 

11.  Tsai-Hong  Hong  and  Azriel  Rosenfeld,  "Multiband 

Pyramid  Linking."  TR-1025,  DAAG-53-76C-1038 , 

March  1981. 

ABSTRACT:  A  method  of  image  segmentation  has  been  developed 

based  on  creating  links  between  pixels  in  successive  layers 
of  a  "pyramid"  of  reduced-resolution  versions  of  the  image. 

In  the  original  implementation  of  this  method,  the  links  were 
based  on  comparing  the  values  of  a  single  feature,  (average) 
gray  level,  for  each  pixel.  In  this  note,  the  method  is 
extended  to  links  based  on  multiple  features,  such  as  cblor 
components  or  neighborhood  properties. 

12.  Cheng-ye  Wang,  Hanfang  Sun ,  Shiro  Yada,  and  Azriel 

Rosenfelu,  "Some  Experiments  in  Relaxation  Image 
Matching  Using  Corner  Features."  TR-1071,  DAAG- 
5  3 - 7 6 C- 0 1 3 8 ,  July  1981. 

ABSTRACT:  A  relaxation  method  based  on  patterns  of  local 

features  is  used  to  find  matches  between  pairs  of  images  or 
subimages  -that  differ  in  position  or  orientation.  A  local 


operator  is  applied  to  the  two  images  to  detect  two  sets  of 
"corners”  and  Di,...,Dn,  each  of  them  characterized 

by  position,  orientation,  contrast,  and  "sharpness"  (of  the 
angle).  For  each  pair  (Ci,Dj),  a  figure  of  merit  is  computed, 
and  a  relaxation  process  is  used  to  iteratively  adjust  these 
figures  of  merit,  based  on  the  merits  of  other  pairs  in  approxi¬ 
mate^  corresponding  positions.  After  a  few  iterations  of 
this  process,  "good"  matches  (pairs  having  much  better  merit 
than  their  next  best  choices)  are  clustered,  yielding  sets  of 
transformation  parameters  (shift  vectors  or  rotation  angles) 
under  which  many  corners  correspond.  This  method  has  yielded 
good  results  for  TV  images  of  objects  such  as  tools  and  indus¬ 
trial  parts,  as  well  as  for  aerial  images  of  terrain. 

13.  Zhong-Quan  Wu  and  Azriel  Rosenfeld,  "Filtered  Project¬ 
ions  as  an  Aid  in  Corner  Detection."  TR-1078, 
DAAG-53-76C-0138,  July  1981. 

ABSTRACT:  Corners  are  very  useful  features  for  such  purposes 
as  image  matching  or  shape  analysis,  but  corner  detection  is 
a  relatively  expensive  operation.  This  paper  uses  filtered 
x  and  y  projections,  applied  to  an  image  containing  an  object 
that  nas  not  been  explicitly  segmented  from  its  background, 
to  determine  possible  positions  of  corners,  so  that  corner 
detection  can  be  applied  only  in  the  vicinity  of  these  posi¬ 
tions.  Even  in  cases  where  the  object  would  be  hard  to  seg¬ 
ment  (unimodal  histogram) ,  this  approach  yields  a  good  set 
of  possible  corner  positions. 

t 

14.  T.  H.  Hong,  M.  Shneier,  and  A.  Rosenfeld,  "Border  Extrac¬ 

tion  Using  Linked  Edge  Pyramids.”  TR-1080,  DAAG-53- 
76C-0138 ,  July  1981. 

ABSTRACT:  Borders  of  objects  in  a  noisy  image  can  be  cleanly 

extracted  by  a  parallel  procedure  involving  links  between  edge 
points  in  a  pyramid  of  reduced-resolution  versions  of  the  image. 

15.  Cheng-Ye  Wang  and  Azriel  Rosenfeld,  "Elevation  Texture." 

TR- 1086 ,  DAAG-53-76C-0138,  August  1981. 

ABSTRACT:  Given  an  array  of  terrain  elevation  values,  texture 

nm  lysis  techniques  can  be  used,  in  principle,  for  segmenta¬ 
tion  and  terrain  type  classification. 

16. -  K.  Prazdny,  "Computing  Motions  of  (Locally)  -Planar 

Surfaces  from  Spatio-Temporal  Chances  in  Image 
Brightness:  A  Note. "  TR-1090,  DAAG-53-76C-0138 , 

August  1981. 

ABSTRACT:  The  motion  at  a  locus  on  the  projection  surface 

cannot  be  obtained  locally  from  spatio-temporal  changes  in 
image  intensity  at  that  locus.  If  the  imaged  surfaces  are 
•locally)  planar,  however,  and  generate  a  set  of  non-parallel 
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straight  line  edges  in  the, image,  it  is  possible  to  obtain 
estimates  of  image  velocities  of  a  set  of  image  points  corres¬ 
ponding  to  the  intersections  of  the  lines.  The  availability 
o:  such  a  (small)  set  is  sufficient  to  resolve  the  underlying 
optical  flow  into  its  translational  and  rotational  components. 


« ) 

17.  Hanfang  Sun,  "Image  Registration  by  Combining  Feature 

Matching  and  Gray  Level  Correlation. "  TR-1091, 

DAAG- 5 3- 76C- 0138,  August  1981. 

ABSTRACT :  A  method  of  image  registration  based  on  matching 

patterns  of  local  features  was  described  in  an  earlier  report. 
Th i s  note  describes  the  use  of  gray  level  correlation  to 
handle  cases  in  which  feature  matching  gives  ambiguous  re¬ 
sults.  Substantial  improvement  in  matching  performance  can 
i  t-  ,-K-h  •  uved  at  little  additional  computational  cost. 

18.  Matti  Pietikainen,  Azriel  Rosenfeld,  and  Larry  S.  Davis, 

"Texture  Classification  Using  Averages  of  Local 
Pattern  Matches."  TR-1098,  DAAG-53-76C-0128 ,  Sep¬ 
tember  1981. 

ABSTRACT:  Laws  has  introduced  a  class  of  texture  features 

based  on  average  degrees  of  match  of  the  pixel  neighborhoods 
with  a  set  of  standard  masks,  l/hese  features  yield  better 
texture  classification  than  standard  features  based  on  pairs 
of  pixels.  This  paper  investigates  simplifications  of  these 
features,  and  shows  that  their  performance  is  not  greatly 
affected  by  their  exact  form,  and  also  appears  to  remain  the 
same  if  only  local  match  maxima  are  used.  It  also  presents 
an  alternative  definition  of  such  features  based  on  sums  and 
rii:ferences  of  Gaussian  convolutions. 


Hnlph  L.  i.artley,  Leslie  J.  Kitchen,  Cheng-Ye  Wang, 
and  Azriel  Rosenfeld,  "A  Comparative  Study  of  Seg¬ 
mentation  Algorithms  for  FLIR  Images."  TR-1104,' 
D/.AG-53-76-C-0138 ,  September  1981. 
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A  comparative  study  of  FLIR  segmentation  algorithms 
.onducted  in  cooperation  with  Westinghouse  Defense 
vision.  In  the  Maryland  portion  of  the  study,  four 
;  and  three-class  relaxation,  "pyramid  linking", 

spike  )  were  tested  on  a  West inghouse-suppl ied  data- 
lmages  obtained  from  NVL  and  other  sources.  (Two 
niques,  superslice"  and  "pyramid  spot  detection", 
ted  after  preliminary  studies.)  The  best  technique, 
e  ,  extracted  regions  correspondinc  to  over  88% 

gets,  and  had  a  false,  alarm  rate  of  1.6  false  regions 
arqet.  ,*  ^ 
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20. 


Stanley  Dunn,  Ludvik  Janos,  and  Azriel  Rosenfeld,  "Bi¬ 
mean  Clustering."  TR-1106,  DAAG-53-76-C-0138 , 

September  1981. 

ABSTRACT:  An  algorithm  is  presented  which  finds  the  best- 
fitting  pair  of  constants,  in  the  least  squares  sense,  to  a 
i'et  of  scalar  data;  we  call  this  pair  of  constants  the  "bimean" 
of  the  data.  The  relationship  of  the  bimean  clustering  to 
the  ISODATA  clustering  algorithm,  and  its  application  to  image 
thresholding,  are  also  discussed. 

21.  T.  II.  Hong  and  M.  Shneier,  "Extracting  Compact  Objects 
Using  Linked  Pyramids."  TR-1123,  DAAG-53-7  6C-0138  , 
November  1981. 

ABSTRACT:  Compact  objects  of  arbitrary  size  are  extracted 

from  images  using  a  combination  of  three  pyramid-based  repre¬ 
sentations  of  image  features.  A  gray-scale  linked  pyramid  is 
used  to  smooth  the  image  into  uniform  regions.  A  "surrounded- 
nuss"  pyramid  is  used  to  identify  regions  of  interest,  and  a 
linked  edge  pyramid  is  u^ed  to  delimit  the  boundaries  of  the 
compact  objects. 

22.  Larry  S.  Davis,  Hanfang  Sun,  and  Zhongquan  Wu,  "Motion 
Detection  at  Corners."  TR-1130,  DAAG-53-76-C-0138 , 

December  1981. 

ABSTRACT:  A  method  for  directly  computing  image  motion  at 

corners  is  presented.  The  method  is  based  on  temporal  inten¬ 
sity  gradients  along  lines  parallel  to  the  sides  of  the  cor¬ 
ner.  The  results  of  applying  the  method  to  two  time-varying 
images  are  discussed. 

23.  Zhongquan  Wu,  Hanfang  Sun,  and  Larry  S.  Davis,  "Determining 
Velocities  by  Propagation."  TR-1132,  DAAG-53-76C-0138 , 
December  1981. 

ABSTRACT:  A  velocity  propagation  technique  is  described 

that  determines  velocity  vectors  at  the  points  of  a  con¬ 
tour,  based  on  the  velocities  at  the  endpoints  of  the  con¬ 
tour  and  the  normal  components  of  velocity  along  the  con¬ 
tour  . 

24.  Tsai-Hong  Hong  and  Azriel  Rosenfeld,  "Unforced  Image 

Partitioning  by  Weighted  Pyramid  Linking."  TR-1137, 

DAAG-5 3-76C-01 38 ,  January  1982. 

ABSTRACT:  This  paper  describes  a  method  of  image  seg¬ 

mentation  that  creates  a  partition  of  the  image  into 
compact,  homogeneous  regions  using  a  parallel,  iterative 
approach  that  does  not  require  immediate  forced  choices. 

The  approach  makes  use  of  a  "pyramid"  of  successively 
reduced-resolution  versions  of.  the  ima.ge.  It  defines 


link  strengths  between  pairs  of  pixels  at  successive 
levels  of  this  pyramid/  based  on  proximity  and  similarity, 
and  iteratively  recomputes  the  pixel  values  and  adjusts 
the  link  strengths.  After  a  few  iterations,  the  link 
strengths  stabilize,  and  the  links  that  remain  strong 

L.es  Kitchen,  "Scene  Analysis  Using  Region-Based  Constraint 
Filtering."  TR-1150,  DAAG-53-76C-0138 ,  February  1982. 

ABSTRACT:  A  general-purpose  scene-analysis'  system  is  de¬ 

scribed  which  uses  constraint-filtering  techniques  to  apply 
domain  knowledge  in  the  interpretation  of  the  regions  ex¬ 
tracted  from  a  segmented  image.  An  example  is  given  of  the 
configuration  of  the  system  for  a  particular  domain,  FLIR 
(Forward  Looking  InfraRed)  images,  as  well  as  results  of 
the  system's  performance  on  some  typical  images  from  this 
domain . 

Cheng- Ye  Wang  and  Leslie  Kitchen,  "Improvements  in  Multispec- 
trnl  Image  Smoothing."  TR-1152,  DAAG-53-76-01 38 ,  March  1982. 

ABSTRACT:  Two  problems  with  the  multispectral  smoothing 
technique  of  Pietikainen  and  Rosenfeld  are  that  random  sampling 
fluctuations  in  the  scatter  plot  may  produce  spurious  pixel 
classes,  and  that  noise  pixels  sufficiently  distinct  from 
choir  neighbors  will  not  be  smoothed  at  all.  These  problems 
can  be  overcome  by  the  use  of  variable-neighborhood  smoothing 
in  the  scatterplot,  and  by  a  judicious  use  of  median  filter- 
in*]  in  conjunction  with  multispectral  smoothing. 

Azriel  Rosenfeld,  "Computer  Vision."  TR-1157 ,  DAAG-53- 
7GC-0133,  April  1982. 

■-.CFVRACT:  Computer  vision  deals  with  the  analysis  and 

interpretation  of  images.  Over  the  past  25  years,  many 
ad  hoc  techniques  for  analyzing  images  have  been  de¬ 
veloped,  but  during  the  past  few  years  this  technology 
has  becrun  to  develop  a  scientific  basis.  This  paper  'i 
outlines  the  major  components  of  a  general  computer 
vision  system,  and  summarizes  the  state  of  the  art  in 
each  of  these  components. 

Fatvi  B.  Boppana  and  Azriel  Rosenfeld,  "Some  Properties  of 
Hueckel-Type  Edge  Operators."  TR-1178,  DAAG-53-76C-0138 , 
June  1982. 

ABSTRACT:  Hueckel-type  edge  operators  find  the  best-fitting 

step  function  to  a  given  image  neighborhood.  This  note 
derives  some  basic  properties  of  such  operators  for  various 
classes  of  neighborhoods. 
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Larry  S.  Davis,  Zhongguan  Wu  and  Hanfang  Sun,  "Contour- 
based  Motion  Estimation."  TR-1179,  DAAG-53-76C-0138 , 

June  1982. 

ABSTRACT:  This  paper  introduces  a  contour-based  approach  to 
motion  estimation.  It  is  based  on  first  computing  motion 
at  image  corners,  and  then  propagating  the 'Corner  motion 
vstimates  along  the  principal  contours  in  the  image  based 
on  a  local  2  1/2D  motion  assumption.  The  results  .of  several 
experiments  are  presented. 

T.  H.  Hong,  M.  Shneier,  R.  Hartley  and  A.  Rosenfeld,  "Using 
Pyramids  to  Detect  Good  Continuation."  TR-1185, 
DAAG-53-76C-0138,  July  1982. 

ABSTRACT:  Pictures  containing  lines  and  curves  are  often 
perceived  by  humans  as  being  composed  of  a  smaller  number  of 
more  global  figures.  For  example,  a  broken  set  of  collinear 
line  segments  is  perceived  as  a  single  straight  line,  even  in 

the  presence  of  other  overlapping  line  segments.  This 
paper  presents  a  method  of  extracting  such  figures  from 
images  automatically  in  a  highly  parallel  manner.  A  pyramid 
of  successively  lower-resolution  images  is  used  to  transform 
the  problem  from  one  of  global  search  to  one  of  local  good 
continuation.  Using  the  pyramid,  figures  that  are  both 
sparse  and  obscured  can  successfully  be  extracted  and  the 
background  clutter  can  be  suppressed. 

Kwangyoen  Wohn,  Larry  S.  Davis  and  Philip  Thrift,  "Motion 

Estimation  Based  on  Multiple  Local  Constraints  and  Nonlinear 
R-oothinq."  TR-U88,  DRAG  53-7600138,  August  1982. 

ABSTRACT:  A  new  algorithm  for  estimating  motion  from  image 

'  •.'U nee s  is  presented.  , .  Initial  motion  estimates  are  determined 
based  on  a  least-squares  solution  to  a  set  of  independent 
linear  constraints  on  the  motion  at  a  pixel.  These  initial 
estimates  are  then  improved  by  a  nonlinear  smoothing  opera¬ 
tion.  The  results  of  this  algorithm  are  compared  with  those 
obtained  by  the  Horn-Schunck  algorithm  [10]  on  a  number  of 
image  sequences. 

Larry  5.  Davis,  Hu-chen  Xie  and  Azriel  Rosenfeld,  "Image 
Sequence  Enhancement',  Based  on  The  Normal  Component  of 
Image  Motion."  TR-1189,  DAAG-53-76C-0138 ,  July  19jB2. 

ABSTRACT:  Huang  and  Hsu  [1]  describe  an  image  sequence 

enhancement  algorithm  based  on  computing  motion  vectors 
between  successive  frames  and  using  these  vectors  to  determine 
the  correspondence  between  pixels  for  frame  averaging.  In 
this  note,  we  demonstrate  that  it  may  be  sufficient  to  use 
only  the  components  of  the  motion  vectors  in  the  gradient 
direction  (called  the  normal  components)  to  perform  the 

enhancement. 
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